Database system with dynamic database caching

ABSTRACT

A fully transactional mid-tier database system services database transactions. A cache manager dynamically loads database entries from a fully transactional backend-tier database system into the mid-tier database system according to the received database transactions. Time based aging or usage based aging can be assigned to selected tables in the mid-tier database system. Database entries contained in the selected tables are then automatically removed according to assigned aging constraints.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to provisional application Ser. No.60/905,751 filed on Mar. 7, 2007, entitled MAIN-MEMORY DATABASES andalso claims priority to provisional application Ser. No. 61/026,090filed on Feb. 4, 2008, entitled DATABASE SYSTEM WITH DYNAMIC DATABASECACHING AND DATABASE SYSTEM WITH ACTIVE AND STANDBY NODES and are bothincorporated by reference in their entirety.

This application is also related to the following application filedsimultaneously herewith and is incorporated by reference in itsentirety.

U.S. patent application Ser. No. 12/030,094 entitled: DATABASE SYSTEMWITH ACTIVE AND STANDBY NODES filed on Feb. 12, 2008.

TECHNICAL FIELD

The present disclosure relates generally to database systems.

BACKGROUND

A disk-based Relational Database Management System (RDBMS) uses diskstorage to store and access large amounts of data. Much of the workperformed by a conventional, disk-optimized RDBMS assumes that dataprimarily resides on disk. Optimization algorithms, buffer poolmanagement, and indexed retrieval techniques are designed based on thisfundamental assumption. One problem with disk storage is that access tothe data is relatively slow.

Even when an RDBMS is configured to hold data in main memory,performance is still hobbled by assumptions of disk-based dataresidency. These assumptions cannot be easily reversed due to hard-codedprocessing logic, indexing schemes, and data access mechanisms.

In-memory resident relational database systems are deployed in theapplication-tier and operate in physical memory using standardSequential Query Language (SQL) interfaces. By managing data in memoryand optimizing data structures and access algorithms, in-memory databasesystems can provide improved responsiveness and throughput compared evento fully cached disk-based RDBMS. For example, the in-memory databasecan be designed with the knowledge that data resides in main memory andcan take more direct routes to data, reducing the length of the codepath and simplifying algorithms and structure.

When the assumption of disk-residency is removed, complexity isdramatically reduced. The number of machine instructions drop, bufferpool management disappears, extra data copies are not needed, and indexpages shrink. The database design becomes simple and more compact, anddata requests are executed faster. However, in-memory database systemscurrently can only operate on a relatively small static portion of thedata contained in a disk-based database system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram showing a database system thatprovides dynamic database caching.

FIG. 2 is a schematic block diagram showing in more detail how a cachemanager caches database entries in a secondary/application-tier databasesystem.

FIG. 3 is a flow diagram explaining operations performed by the cachemanager in FIG. 2.

FIG. 4 is a schematic block diagram showing how the cache manageroperates with cache groups.

FIG. 5 is a flow diagram describing in more detail how dynamic databasecaching operates with cache groups.

FIG. 6 is a schematic block diagram showing how dynamic database cachingselectively ages out database entries.

FIG. 7 is a flow diagram describing usage based aging in more detail.

FIG. 8 is a flow diagram describing time based aging in more detail.

INTRODUCTION

A fully transactional mid-tier database system services databasetransactions. A cache manager dynamically loads database entries from afully transactional backend-tier database system into the mid-tierdatabase system according to the received database transactions. Timebased aging or usage based aging can be assigned to selected tables inthe mid-tier database system. Database entries contained in the selectedtables are then automatically removed according to assigned agingconstraints.

DETAILED DESCRIPTION

FIG. 1 is a schematic representation of a multi-tiered database system.A primary database system 140 can be any conventional fully-relationaldatabase system, such as a disk-based Relational Database ManagementSystem (RDBMS). The primary database system 140 typically uses diskstorage to store and access large amounts of data that in one exampleincludes multiple different tables 144. The primary database system 140is alternatively referred to as a backend database system or abackend-tier database system.

A secondary database system 122 typically operates on a server 100 thatis remote from primary database system 140 and includes a storagemanager that stores and manages different tables 127 that containdifferent database entries. The secondary database 122 in one example isan in-memory fully-relational database that is deployed in anapplication tier and operates in physical memory of the server 100. Thesecondary database system 122 is alternatively referred to as anapplication-tier database system or an in-memory database system.

Applications 112A and 112B are initiated by clients 102A and 102B,respectively, via a local or wide area network 110. The network 110 isalternatively referred to as the Internet. The applications 112 can beany software program that accesses or references database entries in adatabase. For example, the applications 112 could be software programsused for booking airline reservations, ordering products over theInternet, managing financial transactions for banks or investmentinstitutions, or tracking telephone call usage. Of course these are justa few examples of the essentially limitless number of data managementapplications that may be used with the database systems shown in FIG. 1.

Connections from the clients 102 can either be direct connections orclient/server connections. Direct connections refer to Sequential QueryLanguage (SQL) libraries and routines that implement a direct driver.The application 112A can create a direct driver connection when it runson the same server 100 that operates the secondary database system 122.In a direct driver connection, the direct driver directly loads thesecondary database 122 into the application's heap space or a sharedmemory segment. The application 112A then uses the direct driver toaccess a memory image of the secondary database 122. Because nointer-process communication is required, a direct driver connectionprovides fast performance.

The client/server connection accommodates connections from the remoteclient 102B to secondary database 100 over network 110. Applications112B on the client 102B issue calls to local client driver libraries114B that communicate with a server/child process 113 on the server 100containing secondary database 122. The server/child process 113, inturn, issues native requests to the direct driver provided by the serverlibraries for accessing the secondary database 122. If a client 102 andserver 100 reside on separate nodes in a network, then communication isprovided using sockets and Transmission Control Protocol/InternetProtocol (TCP/IP) communications.

The secondary database 122 maintains durability through a combination oftransaction logs 124 and periodic refreshes of a disk-resident versionof the secondary database 122. The transaction logs 124 are written todisk asynchronously or synchronous with the completion of transactions118 and are controlled by the applications 112 at the transaction level.The transaction logs 124 can be used to recover a transaction 118 if theapplication 112 or database 122 fails, undo transactions 118 that arerolled back, replicate changes to other databases, replicate changes inthe secondary database 122 to the primary database 140, or enableapplications 112 to detect changes to database entries.

Checkpoint files 126 are used to keep a snap shot of the secondarydatabase 122. In the event of a system failure, the checkpoint files 126are used to restore the secondary database 122 to a last transactionallyconsistent state. A checkpoint operation scans the secondary database122 for blocks that have changed since the last checkpoint and updatesthe checkpoint files 126 with the changes and removes any transactionlog files 124 that are no longer needed.

The applications 112 create and manage the tables 127 that may existonly in secondary database 122. The applications 112, through cachemanager 150, can also cache frequently used subsets of database entriesfrom the primary database 140. The tables 127 managed exclusively by thesecondary database 122 and the tables 127 cached from primary database140 may all coexist in the same secondary database 122, and are allpersistent and recoverable.

Queries and updates to the tables 127 are performed by the applications112 through standard SQL. Applications 112 running on other differentmid-tier servers may cache different or overlapping subsets of the datain primary database 140.

The cache manager 150 can cache entire tables or table fragments fromthe primary database 140 to the secondary database 122 operating onserver 100. The table fragments are described through an extended SQLsyntax and are cached into corresponding tables. For example, tables128A, 130A, and 132A from primary database 140 are cached intocorresponding tables 128B, 130B, and 132B in the secondary database 122.The cached tables 128B, 130B, or 132B may comprise the entirecorresponding tables 128A, 130A, or 132B from primary database 140 ofmay only include selected database entries from the primary databasetables 128A, 130A, or 132B. The database entries can be any record,tuple, column, row or other data item that typically exists in a fullytransactional database system.

The secondary database 122 dynamically caches performance-criticalsubsets of the primary database 140, enabling both reads and updates,and automatically manages data consistency between the cached secondarydatabase 122 and the primary database 140. The applications 112 read andupdate the cached tables 127 using standard SQL, and the cache manager150 automatically propagates updates from the primary database 140 tothe secondary database 122 and vice versa.

Thus, the cached secondary database 122 offers applications 112 the fullgenerality and functionality of a fully-relational database, thetransparent maintenance of cache consistency with the primary database140, and the real-time performance of an application-tier in-memorydatabase system.

FIG. 2 shows the operations performed by the cache manager 150 in moredetail. The cache manager 150 dynamically varies what subset of tables127 are cached from the primary database system 140 into the secondarydatabase system 122 according to the transactions 118 received from theapplications 112 in FIG. 1.

The cache manager 150 first determines what transactions 118 can beserviced by the secondary database 122. For example, the cache manager150 determines if the referenced tables 200A and referenced primary keys200B in SQL statement 200 reside in secondary database 122. If thereferenced database entries reside in the secondary database 122, thetransaction 118 is serviced by the secondary database 122.

When the database entries referenced by the transaction 118 do notreside in the secondary database 122, the cache manager 150 may querythe primary database 140 for the missing database entries. For example,table identifier 119A and primary key identifier 119B reference adatabase entry 204 in a table 130B having a primary key value PK=1.Since the database entry 204 is not currently located in the secondarydatabase 122, the cache manager 150 queries the primary database 140.The referenced database entry 204 in primary database 140 is theninserted into table 130B in the secondary database 122. The transaction118 may then be serviced by the secondary database 122 using theuploaded database entry 204.

FIG. 3 explains some of the operations performed by the cache manager150 in more detail. Referring both to FIGS. 2 and 3, in operation 250the cache manager 150 receives or monitors the database transactions 118directed to secondary database 122. In operation 252, the cache manager150 may identify the database entries associated with the databasetransaction. For example, the cache manager 150 obtains the tableidentifiers and keys referenced by the transaction 118. The cachemanager 150 in operation 254 searches the secondary database 122 for thereferenced database entries.

If the secondary database 122 contains the referenced database entriesin operation 254, the transaction is serviced by the secondary databasein operation 256. Otherwise, the cache manager 150 sends one or morequeries to the primary database 140 that reference the database entriesthat are not contained in the secondary database 122.

In some embodiments, the secondary database 122 may contain some, butnot all, of the database entries referenced by the transaction 118. Inthis situation, the cache manager 150 may send queries referencing onlythe missing database entries. In other embodiments, when only some ofthe database entries referenced by the transaction 118 are currentlylocated in the secondary database 122, the cache manager 150 may querythe primary database 140 for all of the database entries referenced bythe transaction 118.

The database entries accessed in the primary database 140 are thenuploaded into the secondary database 122 in operation 260. For example,the cache manager 150 may generate additional SQL statements that causethe primary database entries to be inserted into the secondary database122. Any required commitment is performed on the uploaded databaseentries 204 in operation 262. The transaction 118 is then serviced bythe secondary database in operation 256.

Cache Groups

FIG. 4 shows how dynamic database caching is used in conjunction withcache groups. A cache instance or cache group is a collection of relatedrecords that are uniquely identifiable, and is used to model a complexobject. For example, a cache group 314 may be a group of rows thatcorrespond to a set of frequently used tables that are related to eachother through foreign key constraints.

Cache instances/cache groups can be used when both loading data fromprimary database 140 into the secondary database 122 and via versa andwhen database entries are aged out of the secondary database 122. Thecache group 314 may be configured to contain entire tables or configuredto contain only subsets of table rows and/or table columns.

The following SQL syntax is one example of how the cache group 314 iscreated that includes different database entries from both CUSTOMERtable 302 and ORDER table 304.

create cache group cache_customer from customer(pk1 int not null primarykey), orders(pk2 int not null primary key, fk2 int, foreign key (fk2)references customer(pk1));

In this example, each customer in the CUSTOMER table 302 has a primarykey on its ID. One customer may have many orders in the ORDER table 304,where each order has a foreign key (fk2) that references a CUSTOMER(ID).Configuring cache group 314 causes the cache manager 150 to treat all ofthe order information and associated customer information associatedwith the transaction as a single cache instance. For example, atransaction may only reference one of the database entries associatedwith cache group 314. If the referenced database entry is not containedin secondary database 122, the cache manager 150 uploads all of thedatabase entries associated with the cache instance from the primarydatabase 140 at the same time.

In this example, CUSTOMER table 302 is considered a root table andORDERS table 304 is considered a child table. Database entries can beuploaded or flushed based on the root table 302. For example, all childrows for a root table currently located in the secondary database 122can also be presumed to be currently located in the secondary database122. This prevents the cache manager 150 from having to determine if allof the foreign keys for a cache group exist in the secondary database122.

Referring to FIGS. 4 and 5, the cache manager 150 receives a transaction306 in operation 350. The transaction 306 selects all of the orders fora customer having an ID=100. This transaction 306 may be implementedusing the following SQL statement 307.

-   -   select * from orders with a customer_ID=100

The cache manager 150 in operation 352 determines if the secondarydatabase 122 contains any database entries in the ORDERS table 304 withthe customer_ID=100. If so, the SQL statement 307 is serviced by thesecondary database 122 in operation 354 by returning the requesteddatabase entries.

When the secondary database 122 does not contain orders withcustomer_ID=100, the cache manager 150 sends the following query 308 tothe primary database 140 in operation 356 selecting the database entriesfrom the ORDER table with customer_ID=100.

-   -   select * from orders with a customer_ID=100

However, the cache manager 150 also determines that the referenceddatabase entries are part of the cache group 314. The cache manager 150identifies the cache group via the foreign keys assigned to orders intable 304. Accordingly, the cache manager 150 in operation 358 sends thefollowing second query 310 that selects all of the rows from theCUSTOMER table in the primary database 140 that have an ID=100.

-   -   select * from customer where ID=100

The different database entries accessed in the primary database withqueries 308 and 310 are referred to as cache instance 320. It should beunderstood that the two different queries 308 and 310 select moredatabase entries 320A-320C from the primary database 150 than what wasactually referenced by the transaction 306. Thus, in one embodiment, allof the database entries associated with a same cache instance are loadedinto the secondary database at the same time.

The cache manager 150 also sends insert commands 312 to the primarydatabase 140 in operation 360 which cause the rows 320A-320C associatedwith cache instance 320 to be inserted into the secondary database inoperation 360. The transaction 306 is then serviced by the secondarydatabase in operation 354.

Aging

FIG. 6 shows how different aging parameters are used to automaticallyremove database entries from the secondary database 122. Assigningdifferent aging parameters to different database tables allows thesecondary database 122 to dynamically cache more relevant user content.For example, an airlines reservation system may use the secondarydatabase 122 for caching a subset of customer flight reservations andcaching a subset of airline flight schedules.

The customer flight reservation tables may be more effectively cachedbased on usage. For instance, it may be advantageous to maintaincustomer information in the secondary database 122 for customers whofrequently or most recently book airline reservations. This allowsfaster database response to user reservation queries and further mayreduce the amount of traffic between the secondary database 122 andprimary database 140. This is referred to generally as “usage basedaging.”

Other types of data may be more “time based.” For example, airlineflight schedules may be highly queried for some period of time. However,after the airline flight arrives at a destination, that flightinformation is much less likely to be queried again by users.Accordingly, it may be advantageous to remove this type of “time-based”data from the secondary database 122 after a specified time period. Thecache manager 150 can be programmed to selectively associate differenttables in secondary database 122 with these different usage based andtime based aging constraints.

Usage Based Aging

The following SQL, statements may be used for configuring tables A and Cin FIG. 6 as usage based aging tables.

CREATE TABLE A (C1 INT , C2 INT ); ALTER table A ADD AGING LRU; TTAgingLRUConfig (LowUsageThreshold, HighUsageThreshold,AgingCycle)

CREATE TABLE C (C1 INT , C2 INT ); ALTER table C ADD AGING LRU; TTAgingLRUConfig (LowUsageThreshold, HighUsageThreshold,AgingCycle)

A listing 400 identifies in column 400A the tables in secondary database122 that are configured with usage based aging constraints. In thisexample, the Least Recently Used (LRU) database entries in tables A andC are periodically removed according the amount of available space inthe secondary database 122.

High Usage Threshold (HUT) values 400B identify a percentage of usedmemory space in the secondary database 122 that trigger the cachemanager 150 to remove least recently used database entries. Low UsageThreshold (LUT) values 400C can also be assigned to the tables A and Band identify a second lower percentage of used memory space in thesecondary database 122. When the HUT value 400B is reached, the cachemanager 150 removes least recently used database entries until thestorage space in secondary database 122 reaches the LUT value 400C.

Aging Cycle (AC) values 400D in listing 400 indicates how often thecache manager 150 evaluates the least recently used database entries intables A and C. For example, a counter or clock 404 is monitored bycache manager 150. The cache manager 150 periodically checks the amountof used memory space in the secondary database 122 after counter/clock404 indicates the expiration of each aging cycle 400D. If the amount ofused memory space reaches HUT 400B, the cache manager 150 removes theleast recently used database entries from the associated tables A and/orC.

Last Used (LU) tags 408 and 412 indicate when the database entries intables A and C were respectively last used. The LU tags 408 and 412 mayuse the value provided by counter/clock 404 at the time the associateddatabase item was last accessed or referenced. The LU tags 408 and 412can then be updated with a current time from counter/clock 404 wheneverthe associated database entries in tables A or C are accessed orreferenced again by another transaction or when the database entries areuploaded again from the primary database 140.

Time Based Aging

Selected tables in the secondary database 122 can also be assigned timebased aging constraints. For example, the following SQL statementconfigures time based aging constraints for table D.

-   -   CREATE TABLE D (Timestamp C1, Timestamp c2, c3 INT) AGING USE c1        LIFETIME {MINUTES|HOURS|DAYS} CYCLE {MINUTES|HOURS|DAYS}

The time based aging SQL statement causes table D to be listed in column402A of time based aging listing 402. A lifetime value 402B in listing402 designates how long database entries in table D should reside in thesecondary database 122. A cycle time value 402C defines a time periodfor the cache manager 150 to periodically evaluate the database entriesin table D. If different cycle times 402C are defined for differenttables, then the cache manager 150 may wake up based on an a singlecycle time value for all of the tables, or may wake up according to thecycle times for each individual table.

The time based aging SQL statement above also configures a column intable D with timestamp values 414. The timestamp values 414 could be adate and time value from counter/clock 404 or could alternatively be acounter value from counter/clock 404 that is continuously incrementeduntil reaching a reset value. In one embodiment, the timestamp values414 are set to the value of counter/clock 404 when the associateddatabase entries are first loaded into the secondary database 122.

Referring both to FIGS. 6 and 7, in operation 450 the tables in thesecondary database 122 are configured with different usage based agingconstraints and time based aging constraints as described above. Inoperation 452, the cache manager 150 identifies the different usagebased aging tables and time based aging tables as defined in listings400 and 402. In operation 454, the different aging parameters in listing400 are identified for the usage based tables. For example, the cachemanager 150 in operation 454 identifies the high usage thresholds 400B,low usage thresholds 400C and the aging cycles 400D for tables A and C.

In operation 456 the cache manager 150 waits for one of the aging cyclesto be reached for one of the tables A or C. For example, the cachemanager 150 determines when the counter/clock 404 reaches the agingcycle 400D for one of the tables A or C. When an aging cycle is reachedin operation 456, the cache manager 150 determines the amount of memoryspace currently being used in the secondary database 122. If the highusage threshold value 400A is reached for either table A or table C inoperation 458, the least recently used database entries for that tableare removed in operation 462.

For example, the high usage threshold value 400B for table A may be setto 75% and the high usage threshold value 400B for table C may be set to85%. If storage in the secondary database 122 is 80% full when thecounter/clock 404 reaches a next aging cycle time 400D, the leastrecently used database entry in table A is removed in operation 462. TheLU tag value LU=2 indicates that database entry 416 is the leastrecently used entry in table A and is accordingly removed from thesecondary database 122 in operation 462.

In operation 464, the cache manager 150 determines the amount of usedstorage space after the database entry 416 is removed in operation 462.If the percentage of used memory space does not drop below the low usagethreshold value 400C for table A in operation 464, the next leastrecently used database entry is removed from table A in operation 462.Database entries are removed from tables A until the amount of utilizedspace in the secondary database drops below the low usage thresholdvalue in operation 464. A next aging cycle is started for table A inoperation 460 and the cache manager 150 waits for the expiration of thenext aging cycle 400D in operation 456 before conducting the next usagebased purge in operation 458.

If the aging cycles 400D for tables A and C are different, then a sameaging cycle value 400D may be used. If the HUT values 400B for bothtable A and table C are reached at the next common aging cycle, thendatabase entries may be removed from both table A and table C in a roundrobin fashion. Alternatively, different LRU aging sessions may beseparately conducted for tables A and C and LRU database entries foreach table removed independently according to their associated HUTvalues 400B, LUT values 400C and aging cycles 400D.

For example, at the next aging cycle for table C, memory utilization insecondary database 122 may exceed the 85% high usage threshold value400B assigned to table C. Accordingly, least recently used databaseentries are removed from table C in operation 462 until the databasestorage reaches the low usage threshold value 400C for table C. In thisexample, the cache manager 150 removes the least recently used databaseentries 418 (DB entries #1, #3, and #4) from table C in order to reachthe low usage threshold value 400C associated with table C.

Higher priority data in a particular table may be assigned larger highusage threshold values 400B and/or larger low usage threshold values400C. In addition, the aging cycles 400D for high priority data may beset to longer time periods. These larger threshold values 400B, 400C,and 400D cause the cache manager 150 to remove the least recently useddatabase items for those tables less frequently. Thus, the usage basedaging parameters 400 allow automatic customized removal of differenttypes of selectable data from the secondary database 122.

FIG. 8 explains in more detail how the cache manager 150 conducts timebased aging. Referring to FIGS. 6 and 8, any tables having time basedaging constraints are identified in operation 480. In this example,table D is assigned a lifetime value 402B and an associated cycle timevalue 402C in listing 402. The cache manager 150 uses the counter/timer404 in operation 482 to determine when a next cycle time 402C isreached. Any rows in table D that have timestamps 414 exceeding thelifetime value 402B are identified in operation 484 and removed inoperation 486.

For example, the lifetime value 402B for table D may be set to aparticular counter value of say LIFETIME=20. When the cycle time 402C isreached in operation 482, the value for counter/clock 404 is comparedwith the timestamp values 414 in table D. The difference between thevalue of counter/clock 404 and the timestamp values 414 are determinedin operation 484. In this example, the counter 404 may have a currentvalue of 28. The difference between the current value of counter 404(28) and the timestamp value for database entry 420 in table D (TS=7) isgreater than the lifetime value 402B (LIFETIME=20). Accordingly, thedatabase entry 420 is removed from table D in operation 486. Any otherdatabase entries in table D with expired lifetimes are also removed inoperation 486.

Similar to usage based aging, different tables can be assigned differentlifetime values 402B and cycle times 402C. Higher priority data may beassigned larger lifetime values 402B and/or may be evaluated lessfrequently by assigning larger cycle time values 402C.

Tables associated with even higher priority data might not be assignedany aging constraints. For example, table B in FIG. 6 is not assignedany aging constraints. Accordingly, the database entries in table Bremain in the secondary database 122 until replaced by updates from theprimary database 140.

Cache Group Aging

Referring back to both operation 462 in FIG. 7 and operation 486 in FIG.8, database entries associated with the same cache group may be removedaccording to usage based or time based aging constraints. Referring alsoback to FIG. 4, the CUSTOMER table 302 may both be assigned usage basedaging constraints. The high usage threshold may be reached for CUSTOMERtable 302 and the database entry PK1=100 in the CUSTOMER table 302 maybe the least recently used. Accordingly, the entire cache instance 320may be removed in operation 462 in FIG. 7. For example, the rootdatabase entry PK1=100 is removed from the CUSTOMER table 302 and thechild database entries PK2=14 and PK2=20 with referencing foreign keysare removed from the ORDERS table 304.

In another example, usage based aging may be assigned to the childORDERS table 304 in FIG. 4. When storage in secondary database 122reaches the high usage threshold value for ORDERS table 304, databaseentry PK1=14 may be the least recently used.

In one embodiment, all of the root and child database entries for thesame cache instance 320 are removed from the secondary database inoperation 462 in FIG. 7. In another embodiment, cache groups are onlyaged based on the database entries in the root table 302. For example,since database entry PK2=14 is located in child table 304 and notlocated in root table 302, no usage based aging is performed.

For time based aging, database entries associated with the same cachegroup may also be controlled by the root table. For example, timestampsmay only be applied to the database entries in the root CUSTOMER table302 in FIG. 4. Whenever one of the database entries in CUSTOMER table302 resides in the secondary database 122 beyond a specified lifetimevalue, all of the database entries associated with that cache instanceare removed at the same time.

For example in FIG. 4, database entry PK1=100 may reside in thesecondary database 122 beyond a lifetime value assigned to CUSTOMERtable 302. Accordingly, both database entry PK1=100 in CUSTOMER table302 and the other database entries PK2=14 and PK2=20 in ORDERS table 304associated with the same cache instance 320 are removed by the cachemanager 150.

The system described above can use dedicated processor systems, microcontrollers, programmable logic devices, or microprocessors that performsome or all of the operations. Some of the operations described abovemay be implemented in software and other operations may be implementedin hardware.

For the sake of convenience, the operations are described as variousinterconnected functional blocks or distinct software modules. This isnot necessary, however, and there may be cases where these functionalblocks or modules are equivalently aggregated into a single logicdevice, program or operation with unclear boundaries. In any event, thefunctional blocks and software modules or features of the flexibleinterface can be implemented by themselves, or in combination with otheroperations in either hardware or software.

Having described and illustrated the principles of the invention in apreferred embodiment thereof, it should be apparent that the inventionmay be modified in arrangement and detail without departing from suchprinciples. Claim is made to all modifications and variation comingwithin the spirit and scope of the following claims.

1. A method, comprising: operating a fully transactional mid-tierdatabase; receiving one or more database transactions at the mid-tierdatabase; and dynamically loading database entries from a fullytransactional backend-tier database into the mid-tier database accordingto the received database transactions.
 2. The method according to claim1 further comprising operating the mid-tier database as an in-memorydatabase.
 3. The method according to claim 1 further comprising:identifying database entries associated with the database transactions;servicing the database transactions with the mid-tier database when theidentified database entries are contained in the mid-tier database; andquerying database entries in the backend-tier database when theidentified database entries are not contained in the mid-tier database.4. The method according to claim 3 further comprising: uploading thequeried database entries into the mid-tier database; and servicing thedatabase transactions with the mid-tier database using at least some ofthe uploaded database entries from the backend-tier database.
 5. Themethod according to claim 1 further comprising: identifying tables andprimary keys referenced by the database transactions; searching themid-tier database for the identified tables and primary keys; servicingthe database transactions with the mid-tier database when the identifiedtables and primary keys are located in the mid-tier database; andaccessing the backend-tier database when the identified tables andprimary keys are not located in the mid-tier database.
 6. The methodaccording to claim 1 further comprising: associating database entriesfrom different tables with a same cache group; identifying a databasetransaction that references one or more database entries associated withthe cache group; uploading the entire cache group from the backend-tierdatabase into the mid-tier database when the referenced database entriesare not contained in the mid-tier database.
 7. The method according toclaim 6 further comprising: identifying a first database entry in afirst table; identifying a primary key associated with the firstdatabase entry in the first table; identifying a second database entryin a second table having a foreign key referencing the primary key inthe first table; and associating both the first database entry in thefirst table and the second database entry in the second table with thesame cache group.
 8. A method, comprising: operating a database system;assigning aging parameters to selected tables in the database system;and removing database items from the selected tables in the databasesystem according to the assigned aging parameters.
 9. The methodaccording to claim 8 further comprising: associating time based agingwith the selected tables in the database system; and removing thedatabase items from the selected tables according to how long thedatabase items have resided in the database system.
 10. The methodaccording to claim 8 further comprising: associating usage based agingwith the selected tables in the database system; and removing thedatabase items from the selected tables according to how recently thedatabase items have been used in the database system.
 11. The methodaccording to claim 10 further comprising: assigning a high usagethreshold value, a low usage threshold value, and an aging cycle to theselected tables in the database system; identifying an amount ofavailable space in the database system for each aging cycle; removing atleast some of the least recently used database items from the selectedtables when an amount of storage space in the database system reachesthe high usage threshold value; and removing least recently useddatabase items from the selected tables until the amount of storagespace in the database system reaches the low usage threshold value. 12.The method according to claim 8 further comprising: configuringdifferent time based aging tables and usage based aging tables in thedatabase system; removing at least some of the database items from thetime based aging tables that have resided in the database system beyonda configured time period; removing at some of the least recently useddatabase items from the usage based aging tables when storage in thedatabase system reaches a configured usage threshold; and skippingremoval of database items from non-time based and non-usage based tablesin the database system.
 13. The method according to claim 8 furthercomprising: associating some of the database items from differentselected tables with a same cache group; assigning an aging parameter toa root table of the cache group; and removing the database items fromboth the root table and one or more child tables associated with thecache group according to the aging parameter assigned to the root table.14. Computer readable media containing instructions that when executedby a computer, comprise: operating a fully-transactional secondarydatabase; receiving database transactions directed to the secondarydatabase; searching the secondary database for data items referenced bythe transactions; servicing the database transactions with the secondarydatabase when the secondary database contains the referenced data items;querying a fully-transactional primary database when the referenced dataitems are not contained in the secondary database; and updating thesecondary database with at least some of the data items queried in theprimary database.
 15. The computer readable media according to claim 14further comprising instructions that when executed result in: assigningaging parameters to selectable data items in the secondary database; andautomatically removing data items from the secondary database accordingto the assigned aging parameters.
 16. The computer readable mediaaccording to claim 14 further comprising instructions that when executedresult in: assigning either a time based aging parameter or a leastrecently used aging parameter to programmably selectable tables in thesecondary database; and removing the data items according to the timebased aging parameter or least recently used aging parameter assigned tothe tables containing the data items.
 17. The computer readable mediaaccording to claim 14 further comprising instructions that when executedresult in: configuring time based aging tables and usage based agingtables in the secondary database; removing at least some of the dataitems from the time based aging tables that have resided in thesecondary database beyond a configured time period; and removing at someof the least recently used data items from the usage based aging tableswhen storage in the secondary database reaches a configured usagethreshold value.
 18. The computer readable media according to claim 14further comprising instructions that when executed result in:associating at least some of the data items from different tables with asame cache group; identifying transactions referencing one or more ofthe data items from the cache group; uploading the entire cache groupfrom the primary database into the secondary database when any of thereferenced data items are not contained in the secondary database. 19.The computer readable media according to claim 18 further comprisinginstructions that when executed result in: assigning an aging parameterto a root table for the cache group; and removing all of the data itemsassociated with the cache group according to the aging parameterassigned to the root table.
 20. A database management system,comprising: an application-tier database system receiving requests fromdatabase applications; and a cache manager configured to maintain atleast a sub-set of database entries from a backend-tier database systemin the application-tier database system, the cache manager dynamicallyvarying what database entries are loaded from the backend-tier databasesystem into the application-tier database system according to therequests from the database applications.
 21. The database managementsystem according to claim 20 wherein the cache manager is furtherconfigured to: monitor the different requests received by theapplication-tier database system; identify what database entries arereferenced by the requests; service the requests with theapplication-tier database system when the referenced database entriesare contained in the application-tier database system; temporarily stallthe requests when the referenced database entries are not contained inthe application-tier database system; query the backend-tier databasesystem for the referenced database entries; load the database entriesqueried from the backend database system into the application-tierdatabase system; and service the requests using at least some of thedatabase entries loaded into the application-tier database system fromthe backend-tier database system.
 22. The database management systemaccording to claim 20 wherein the cache manager is further configuredto: identify cache groups that include both a root table with one ormore database entries having primary keys and one or more child tableshaving one or more database entries having foreign keys referencing theprimary keys in the root table; and dynamically removing the identifiedcache groups from the application-tier database system or dynamicallyloading the identified cache groups from the backend-tier databasesystem into the application-tier database system.
 23. The databasemanagement system according to claim 20 wherein usage based aging isassigned to selected tables in the application-tier database system andthe cache manager removes database entries contained in the selectedtables according to how recently the database entries have been used inthe application-tier database system.
 24. The database management systemaccording to claim 20 wherein time-based aging is assigned to selectedtables in the application-tier database system and the cache managerremoves database entries contained in the selected tables according tohow long the database entries have resided in the application-tierdatabase system.